Prompt Generator for Stable Diffusion
Contents
Prompt Generator for Stable Diffusion#
Note
Install ekorpkit package first.
Set logging level to Warning, if you don’t want to see verbose logging.
If you run this notebook in Colab, set Hardware accelerator to GPU.
!pip install ekorpkit[art] exit()
Preparing the environment#
%config InlineBackend.figure_format='retina'
%load_ext autotime
from ekorpkit import eKonf
eKonf.setLogger("INFO")
eKonf.set_cuda(device=0)
print("version:", eKonf.__version__)
is_colab = eKonf.is_colab()
print("is colab?", is_colab)
if is_colab:
eKonf.mount_google_drive()
project_dir = eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")
print("project_dir:", project_dir)
INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:0)']
INFO:ekorpkit.utils.notebook:Google Colab not detected.
INFO:ekorpkit.base:Setting EKORPKIT_WORKSPACE_ROOT to /content/drive/MyDrive/workspace/
INFO:ekorpkit.base:Setting EKORPKIT_PROJECT to ekorpkit-book
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
version: 0.1.40.post0.dev5
is colab? False
project_dir: /content/drive/MyDrive/workspace/projects/ekorpkit-book
time: 2.16 s (started: 2022-11-04 03:30:54 +00:00)
Load the Generator and Generate Prompts#
from ekorpkit.models.art.prompt import PromptGenerator
pgen = PromptGenerator(verbose=False)
2022-11-04 03:30:57.641122: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
INFO:ekorpkit.batch:Batch name: prompt-batch, Batch num: 1
time: 4.23 s (started: 2022-11-04 03:30:56 +00:00)
prompts = pgen.generate_prompts(prompt="examining documents")
prompts
INFO:ekorpkit.batch:Using existing path: /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen
INFO:ekorpkit.batch:Batch name: prompt-batch, Batch num: 2
INFO:ekorpkit.batch:Merging config with args: {}
INFO:ekorpkit.batch:Setting seed to 1772383741
INFO:ekorpkit.batch:Saving config to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/stable-prompt/prompt-batch/configs/prompt-batch(2)_config.yaml
['examining documents, artstation trends beautiful digital anime painting of a happy cyborg child surrounded by bright cute cute little bubbles bursting out of the clouds with glowing lights, dramatic lighting, intricate, highly detailed, unreal engine, Trend',
'examining documents. There is also a detailed matte painting of a village in the middle. art station, concept art, matte, sharp focus, illustration.. by michael welan, raymond swanland, and greg rut',
'examining documents in an enchanted room surrounded by colorful glowing orbs, dark and ominous mood, volumetric lights, red and cyan theme, cinematic lighting, high quality, highly detailed, octane render, 8 k resolution, trending on artstation',
'examining documents. holographic. photorealistic. highly detailed. cinematic. 4 k closeup portrait shot of beautiful pale gothic girl in white dress and necklaces surrounded by waterfalls with glowing butterflies, intricate,',
'examining documents and videos, trending on artstation, 8k, HD portrait of danny devito smoking a cigar, intricate, elegant, glowing lights, highly detailed, digital painting, artstation, glamor pose,',
'examining documents. intricate elegant. Trending on artstation. 8k a hyper detailed matte painting of a city landscape inside of a gigantic blackberry bushes. at night. dense foliage. aerial photography. extreme bokeh.',
'examining documents and schematics, highly detailed, octane render, 8 k ultra realistic illustration of beautiful female japanese dragon princess, flowing hair, cyberpunk, intricate elegant ornate details, by alphonse',
'examining documents, hyper detailed, ultra realistic, concept art, intricate details, eerie, highly detailed, photorealistic, octane render, 8 k, unreal engine. art by artgerm and greg rutkowski and charlie',
'examining documents and detailed white background, elegant detailed digital painting, concept art, matte, sharp focus, 4k ultra hd, illustration, magical, unreal engine, octane render, cinematic lighting, wide angle establishing shot, extremely high detail',
'examining documents and document insert an emoji with or without spaces, in a cyberpunk city, by Guy Denning, Johannes Itten, Derek Gores, Russ Mills, glitch art, octane render, iridescent accents']
time: 449 ms (started: 2022-11-04 03:52:24 +00:00)
Train a Generator#
pgen.train(model_name="ekorpkit/stable-prompts-2")
WARNING:datasets.builder:Using custom data configuration Gustavosta--Stable-Diffusion-Prompts-d22aeec0ba2a9fdb
WARNING:datasets.builder:Reusing dataset parquet (/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/cache/Gustavosta___parquet/Gustavosta--Stable-Diffusion-Prompts-d22aeec0ba2a9fdb/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)
Token indices sequence length is longer than the specified maximum sequence length for this model (2613888 > 1024). Running this sequence through the model will result in indexing errors
***** Running training *****
Num examples = 18847
Num Epochs = 1
Instantaneous batch size per device = 1
Total train batch size (w. parallel, distributed & accumulation) = 8
Gradient Accumulation steps = 1
Total optimization steps = 2356
Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
ERROR:wandb.jupyter:Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.
wandb: Currently logged in as: entelecheia. Use `wandb login --relogin` to force relogin
wandb version 0.13.4 is available! To upgrade, please run:
$ pip install wandb --upgrade
Tracking run with wandb version 0.13.3
Run data is saved locally in
/workspace/projects/ekorpkit-book/ekorpkit-book/docs/lectures/aiart/wandb/run-20221102_081028-3heixjgo
[2356/2356 04:59, Epoch 1/1]
| Step | Training Loss |
|---|---|
| 200 | 3.776300 |
| 400 | 2.705100 |
| 600 | 2.517600 |
| 800 | 2.376600 |
| 1000 | 2.295400 |
| 1200 | 2.231800 |
| 1400 | 2.175900 |
| 1600 | 2.143700 |
| 1800 | 2.104900 |
| 2000 | 2.082700 |
| 2200 | 2.081700 |
Training completed. Do not forget to share your model on huggingface.co/models =)
Waiting for W&B process to finish... (success).
Run history:
| train/epoch | ▁▂▂▃▄▄▅▆▆▇▇█ |
| train/global_step | ▁▂▂▃▄▄▅▆▆▇▇█ |
| train/learning_rate | █▇▇▆▅▄▄▃▂▂▁ |
| train/loss | █▄▃▂▂▂▁▁▁▁▁ |
| train/total_flos | ▁ |
| train/train_loss | ▁ |
| train/train_runtime | ▁ |
| train/train_samples_per_second | ▁ |
| train/train_steps_per_second | ▁ |
Run summary:
| train/epoch | 1.0 |
| train/global_step | 2356 |
| train/learning_rate | 0.0 |
| train/loss | 2.0817 |
| train/total_flos | 4924659858407424.0 |
| train/train_loss | 2.38621 |
| train/train_runtime | 353.5289 |
| train/train_samples_per_second | 53.311 |
| train/train_steps_per_second | 6.664 |
Synced ekorpkit/stable-prompts-2: https://wandb.ai/entelecheia/ekorpkit-book-prompt-generator/runs/3heixjgo
Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
Find logs at:
./wandb/run-20221102_081028-3heixjgo/logsINFO:ekorpkit.models.art.prompt:Saving model to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2
Configuration saved in /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/config.json
Model weights saved in /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/pytorch_model.bin
INFO:ekorpkit.batch:Saving config to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/stable-prompt/prompt-batch/prompt-batch(14)_config.yaml
time: 7min 9s (started: 2022-11-02 08:09:17 +00:00)
pgen.load_model(model_name="ekorpkit/stable-prompts-2")
INFO:ekorpkit.models.art.prompt:Loading model from /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2
loading configuration file /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/config.json
Model config GPT2Config {
"_name_or_path": "/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2",
"_num_labels": 1,
"activation_function": "gelu_new",
"architectures": [
"GPT2LMHeadModel"
],
"attn_pdrop": 0.1,
"bos_token_id": 50256,
"embd_pdrop": 0.1,
"eos_token_id": 50256,
"id2label": {
"0": "LABEL_0"
},
"initializer_range": 0.02,
"label2id": {
"LABEL_0": 0
},
"layer_norm_epsilon": 1e-05,
"model_type": "gpt2",
"n_ctx": 1024,
"n_embd": 768,
"n_head": 12,
"n_inner": null,
"n_layer": 6,
"n_positions": 1024,
"reorder_and_upcast_attn": false,
"resid_pdrop": 0.1,
"scale_attn_by_inverse_layer_idx": false,
"scale_attn_weights": true,
"summary_activation": null,
"summary_first_dropout": 0.1,
"summary_proj_to_labels": true,
"summary_type": "cls_index",
"summary_use_proj": true,
"task_specific_params": {
"text-generation": {
"do_sample": true,
"max_length": 50
}
},
"torch_dtype": "float32",
"transformers_version": "4.21.2",
"use_cache": true,
"vocab_size": 50260
}
loading weights file /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/pytorch_model.bin
All model checkpoint weights were used when initializing GPT2LMHeadModel.
All the weights of GPT2LMHeadModel were initialized from the model checkpoint at /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2.
If your task is similar to the task the model of the checkpoint was trained on, you can already use GPT2LMHeadModel for predictions without further training.
time: 1.42 s (started: 2022-11-02 08:04:02 +00:00)